Method and system for a chat box eco-system in a federated architecture

ABSTRACT

A method and a virtual agent system services a user request from a user. The virtual agent system includes: (a) a conversational user interface receiving the user request and communicating with two or more virtual agents; and (b) a dialog manager including a natural language processing module, that directs operations of the conversational user interface, wherein the dialog manager (i) receives and analyzes the user request from the conversation user interface using the natural language processing module, (ii) causes the conversational user interface to request and to receive a response to the user request from each of the virtual agents, and (iii) integrates the received responses to the user request into an integrated response based on the natural language processing module and causes the conversational user interface to provide the integrated response to the user.

CROSS REFERENCE TO RELATED APPLICATION

This application relates to and claims priority of U.S. provisionalApplication (“Provisional Application”), Ser. No. 62/637,367, entitled“Method and System for Chatbox Eco System in a Federated Architecture,”which was filed Feb. 8, 2018. This Application is also related to (i)U.S. patent application (“Copending Application I”), Ser. No.15/677,319, entitled “Method and System for Semi-Supervised Learning ingenerating Knowledge for Intelligent Virtual Agents,” filed on Aug. 15,2017 and published as U.S. Patent Application Publication 2018/0053119A1; and (ii) U.S. patent application (“Copending Application II”), Ser.No. 15/677,275, entitled “Method and System for CollaborativeIntelligent Virtual Agents,” filed on Aug. 15, 2017 and published asU.S. Patent Application Publication 2018/0054464 A1.

The Provisional Application and Copending Applications I and II arehereby expressly incorporated by reference in their entireties.

BACKGROUND OF THE INVENTION 1. Technical Field

The present teaching generally relates to online services. Morespecifically, the present teaching relates to methods, systems, andprogramming for virtual agents.

2. Discussion of the Related Art

With the new wave of artificial intelligence (AI), some research efforthas been directed to conversational information systems. Intelligentassistants with a conversational user interface, or so-calledintelligent chat robots (“chat bots”), have emerged in recent years.Examples include Siri® of Apple, Facebook Messenger, Amazon Echo, andGoogle Assistant.

Conventional chat bot systems require many hand-crafted rules andmanually labeled training data for the systems to learn thecommunication rules for each specific domain. This led to expensivehuman-labeling efforts and, hence, high costs. Furthermore, even forhumans, knowledge in each specific domain are often incomplete, notwell-understood or not well-characterized, such that chat bots createdunder different sets of hand-crafted rules and training data may excelin performance under different circumstances, but with none excelingunder all instances.

Therefore, as many chat bots serving a specific domain may becommercially or otherwise available, it would be potentiallyadvantageous to derive from their outputs the best response for a givenset of circumstances.

SUMMARY

According to one embodiment of the present invention, a virtual agentsystem servicing a user request from a user, includes (a) aconversational user interface receiving the user request andcommunicating with two or more virtual agents; and (b) a dialog managerincluding a natural language processing module, that directs operationsof the conversational user interface, wherein the dialog manager (i)receives and analyzes the user request from the conversation userinterface using the natural language processing module, (ii) causes theconversational user interface to request and to receive a response tothe user request from each of the virtual agents, and (iii) integratesthe received responses to the user request into an integrated responsebased on the natural language processing module and causes theconversational user interface to provide the integrated response to theuser.

According to one embodiment of the present invention, in a virtual agentsystem servicing a user request from a user, a method includes: (a) in aconversational user interface, receiving the user request andcommunicating with two or more virtual agents; and (b) in a dialogmanager that includes a natural language processing module and thatdirects operations of the conversational user interface, (i) receivingand analyzing the user request from the conversation user interfaceusing the natural language processing module, (ii) causing theconversational user interface to request and to receive a response tothe user request from each of the virtual agents, and (iii) integratingthe received responses to the user request into an integrated responsebased on the natural language processing module and causes theconversational user interface to provide the integrated response to theuser.

Additional novel features will be set forth in part in the descriptionwhich follows, and in part will become apparent to those skilled in theart upon examination of the following and the accompanying drawings ormay be learned by production or operation of the examples. The novelfeatures of the present teachings may be realized and attained bypractice or use of various aspects of the methodologies,instrumentalities and combinations set forth in the detailed examplesdiscussed below.

BRIEF DESCRIPTION OF THE DRAWINGS

The methods, systems or programming described herein are furtherdescribed in terms of exemplary embodiments. These exemplary embodimentsare described in detail with reference to the drawings. Theseembodiments are non-limiting exemplary embodiments, in which likereference numerals represent similar structures throughout the severalviews of the drawings, and wherein:

FIG. 1 depicts a generalized framework for service agent development andapplication;

FIG. 2 depicts an exemplary high-level system diagram of service virtualagent 142 of FIG. 1.

FIG. 3A shows system 180, which allows the user to interact with one ormore external virtual agents, in accordance with one embodiment of thepresent invention.

FIGS. 3B and 3C are examples of integrating an external virtual agent insystem 160.

FIG. 4A is a flow chart illustrating the operations of a system usingfederated conversational agents.

FIG. 4B is a flow chart further illustrating virtual agent selectionunder step 402 of FIG. 4A.

FIG. 5A illustrates one example of a message-passing network ofdecentralized conversational agents (“Pure P-2-P network”).

FIG. 5B illustrates a “central Hub network,” which is an alternative tothe Pure P-2-P network of FIG. 5A.

FIG. 5C illustrates a hierarchical or “super-peer” network, which isstill another alternative to the Pure P-2-P and Central hub networks ofFIGS. 5A and 5B.

To facilitate cross-referencing among the figures and to simplify thedetailed description below, like elements in the figures are assignedlike reference numerals to the extent practicable.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

In the following detailed description, details are set forth by way ofexamples to provide a thorough understanding of the relevant teachings.However, the present teachings may be practiced without such details. Inother instances, well-known methods, procedures, components, orcircuitry have been described at a relatively high-level, withoutdetail, in order to avoid unnecessarily obscuring aspects of the presentteachings.

The present disclosure generally relates to systems, methods, medium,and other implementations directed to various aspects of technologiesassociated with technologies used in artificial intelligence-basedhuman-machine interactions. In some embodiments, unsupervised,semi-supervised and fully supervised approaches are disclosed forlearning from past and present conversations so as to efficiently andeffectively derive different types of dialog models, includingFrequently-Asked-Questions (“FAQ”) models and task-based conversationmodels. In a different embodiment, to handle dynamically changingcontexts in human-machine conversations, the present teaching alsodiscloses means to automatically selecting and switching resources,adaptive to the dynamic conversation contexts, thereby appropriatelysupporting the changing dialogs. The adaptive selection and switchingresources may include switching from one set of service agents toanother set of service agents based on dynamically developedconversation situations, whether from one or more virtual agents to adifferent set of virtual agents or to a human agent. In some of theembodiments, the virtual agents available to a given virtual agentdevelopment and application system may include virtual agents that areinternal, external, or both, to that given system.

In other embodiments, the present teaching discloses developing,training, and deploying a system having multiple effective intelligentvirtual agents, whether internal or external, that are involvedsimultaneously. In different embodiments, the present teaching disclosesa virtual agent that can carry on a dialog with a user, based onknowledge taught from a bot design programming interface, whileinteracting with multiple virtual agents that provide the user'srequested service. The disclosed system in the present teaching canintegrate the output responses of various intelligent virtual agents togenerate high-quality automatic responses that can be used to assist ahuman representative or agent, to accomplish complex service tasks, orto address a user's information need in an efficient way. Morespecifically, based on machine learning and AI technique, the disclosedsystem can learn how to strategically select or integrate outputresponses from multiple virtual agents to provide a response to a user'srequest, optimized for the current set of circumstances. The disclosedsystem can also learn, build or adapt its ability to handle future userdemands based on the knowledge base and historical conversations tobetter select or integrate of virtual agent responses in futureconversations, using both supervised and unsupervised machine learningor training techniques.

The terms “service virtual agent”, “virtual agent”, “conversationalagent”, “agent”, “bot” and “chat bot” may be used interchangeablyherein. Additional novel features will be set forth in part in thedescription which follows, and in part will become apparent to thoseskilled in the art upon examination of the following and theaccompanying drawings or may be learned by production or operation ofthe examples. The novel features of the present teachings may berealized and attained by practice or use of various aspects of themethodologies, instrumentalities and combinations set forth in thedetailed examples discussed below.

FIG. 1 depicts a generalized framework of the development andapplications of service virtual agents. In this example, the disclosedsystem may include a natural language understanding-based (“NLU-based”)user intent analyzer 120, service agent router 125, N service virtualagents 140, databases 130, and virtual agent development engine 170.

Service virtual agents 140 in FIG. 1 may perform direct dialogs with theusers 110. Each virtual agent may focus on a specific service or domainwhen chatting with one or more users. For example, a user may sendutterances to NLU-based user intent analyzer 120. Upon receiving anutterance from a user, NLU-based user intent analyzer 120 may analyzethe user's intent based on an NLU model and the utterance. In oneembodiment, the NLU based user intent analyzer 120 may utilize machinelearning technique to train the NLU model based on real and simulateduser-agent conversations as well as contextual information of theconversations. NLU-based user intent analyzer 120 may estimate the userintent and send the estimated user intent to service agent router 125for agent routing.

Service agent router 125 in this example may receive the estimated userintent from NLU-based user intent analyzer 120 and may select one ofservice virtual agents 140, based on the estimated user intent. Servicevirtual agents 140 may include, for example, a virtual customer serviceagent, a virtual sales agent, a virtual travel agent, a virtualfinancial advisor, and a virtual sport commenter. Once service agentrouter 125 determines that a service virtual agent has a domain orservice matching the estimated user intent, service agent router 125 canroute the user's utterance to the corresponding virtual agent to enablea conversation between the virtual agent and the user.

During the conversation between the virtual agent and the user, thevirtual agent can analyze and determine dialog states based on thedialog and manage real-time tasks related to the dialog, based on datastored in various databases (e.g. knowledge database 134, a publisherdatabase 136, and a customized task database 139). The virtual agent mayalso perform product or service recommendation to the user based on userdatabase 132. In one embodiment, when the virtual agent determines achange in the user's intent or the user is dissatisfied with the currentdialog, the virtual agent may redirect the user to a different agentbased on virtual agent database 138. The different agent may be adifferent virtual agent or human agent 150. For example, when thevirtual agent detects that the user is asking for a sale related to alarge quantity or a large amount of money (e.g. higher than apredetermined threshold), the virtual agent can escalate theconversation to human agent 150, such that human agent 150 can take overthe conversation with the user. The escalation may be seamless and notcausing any delay to the user.

Virtual agent development engine 170 in this example provides a botdesign programming interface through which a developer may develop acustomized virtual agent. In fact, virtual agent development engine 170can work with multiple developers 160 at the same time, with eachdeveloper developing separately a customized virtual agent with aspecific service or domain. As such, a service virtual agent (e.g. theservice virtual agent 142) may have different versions as shown in FIG.1, each of which corresponds to a customized version generated based ona developer's specific request or specific parameter values. Virtualagent development engine 170 may also store the customized tasks intocustomized task database 139, which can provide previously generatedtasks as a template for future task generation or customization duringvirtual agent development.

FIG. 2 depicts an exemplary high-level system diagram of virtual agent142 of FIG. 1. Virtual agent 142 in this example includes dynamic dialogstate analyzer 210, dialog log database 212, one or more deep learningmodels 225, customized FAQ generator 220, customized FAQ database 222,various databased (e.g., knowledge database 134, publisher database 136,. . . , and customized task database 139), real-time task manager 230,machine utterance generator 240, recommendation engine 250, and agentre-router 260.

Although FIGS. 1 and 2 provide as examples such databases as customizedFAQ database 222, knowledge database 134, publisher database 136, andcustomized task database 139, other databases can be made available fordeveloping virtual agents of specific domains or applications. Suchother databases may be, for example, customer business process databasesand customer relationship modules (CRMs), for virtual agents of variousbusiness applications. Also, customized skill or intent database may beprovided for downstream virtual assistant modules, such as Amazon Alexaskills or Microsoft LUIS customized intents. For virtual agentsassociated with document depository or servers, a database holding acustomer's enterprise document information (e.g. internal or enterprisewebsite pages, enterprise presentation documents, slides and WORDdocuments) may be provided. In general, to support virtual agents fordomain knowledge-specific applications, customized domain-specificdatabases are provided (e.g. drug stock and price databases).

In operation, dynamic dialog state analyzer 210 continuously receivesand analyzes input utterances and data from user 110 and determines theresulting dialog state based on that dialog with user 110. The analysisof the user's input utterances may be achieved via natural languageprocessing (NLP), which can be a key component of dynamic dialog stateanalyzer 210. Different NLP techniques may be employed to analyze theinput utterances from a user. The determination of a dialog state can bebased on, for example, deep learning models 225, and optionally someknown customer-related FAQs from customized FAQ database 222.

Dynamic dialog state analyzer 210 record dialog logs, including both thedialog states and other metadata related to the dialog, into dialog logdatabase 212, which can be used by customized FAQ generator 220 forfurther generating customized FAQs. Dynamic dialog state analyzer 210may also estimate user intent based on the dialog states determined byanalyzing the received user input. The estimated user intent is thensent to real-time task manager 230 for real-time task management.

As discussed herein, dynamic dialog state analyzer 210 may analyze theuser input based on customized FAQ data obtained from customized FAQdatabase 222 generated by customized FAQ generator 220. Customized FAQgenerator 220 in this example may generate FAQ data customized for thedomain associated with virtual agent 142, or customized based on acustomers' specific requirements. For example, when virtual agent 142 isa virtual sales agent, the customized FAQ generator 220 may generateFAQs relevant to sales. Examples of FAQs customized for a sale agentinclude: What products are you selling? What is the price list for theproducts being sold? How can I pay for a product? How much is theshipping fee? How long will be the shipping time? Is there any localstore? The customized FAQ generator 220 may generate these customizedFAQs based on information from different sources such as knowledgedatabase 134, publisher database 136, customized task database 139, andany of the domain knowledge-specific databases.

Information from different sources may provide knowledge of differentperspectives for a virtual agent to utilize. For example, knowledgedatabase 134 may provide information about general knowledge related toproducts and services. Publisher database 136 may provide informationabout each publisher (e.g., products or services the publisher isselling for which companies, what advertisements of which products orservices the publisher is displaying, or which virtual agent thepublisher has deployed to provide services). Customized task database139 may store data related to customized tasks generated according tosome customers' specific requests. For example, if virtual agent 142 isa customized version of a virtual car sales agent developed based on aspecific request for a location having a specific climate (e g , manysnow storms), the customized tasks database generated by customized FAQgenerator 220 may include FAQs customized specifically for that type ofclimate, e.g.: Do you like to add snow tires on your car? Which carshave all-wheel-drive functions? The answers to such questions may alsobe generated by customized FAQ generator 222 based on, e.g., theinformation from knowledge database 134. Such generated customizedquestions or answers may be stored in customized FAQ database 222, whichcan then be retrieved by dynamic dialog state analyzer 210 forunderstanding the user input or by real-time task manager 230 fordetermining how to handle the questions from the user.

The questions and answers stored in customized FAQ database 222 may alsobe used, by customized FAQ generator 220 to generate more customizedFAQs. For example, question “Which cars have all-wheel-drive functions?”may be asked in different ways, including “Do you have any car withall-wheel-drive function,” “How many cars do you have that haveall-wheel-drive function?” Variations of a known question may be a basisfor generating additional customized FAQ questions. The same can beapplied to generating answers to different questions. In this way, thevirtual agent automatically and adaptively continues to enhance itsability to handle more diversified questions.

Customized FAQ generator 220 may also generate customized FAQs based ondata obtained from the dialog log database 212. For example, based onlogs of previous dialogs between virtual agent 142 and various users,customized FAQ generator 220 may identify which question is asked veryfrequently and which question is asked infrequently. Based thefrequencies of the questions asked in the logs, the customized FAQgenerator 220 may generate or update FAQs accordingly in customized FAQdatabase 222. Customized FAQ generator 220 may also send the customizedFAQ data to real-time task manager 230 for determining next task type.

The disclosed system may also include an offline conversation dataanalysis component, which can mine important statistical information andfeatures from historical conversation logs, human action logs and systemlogs. The offline conversation data analysis component, not shown, maybe either within or outside virtual agent 142. The important statisticalinformation and signals (e.g. the frequency of each types of questionand answer, and the frequency of human-edits for each question, etc.)can be used by other system components (such as customized FAQ generator220 for identifying important new FAQs, and the recommendation engine250 for performing high-quality recommendations for products andservices,) for their addressed specific tasks for the disclosed system.

Real-time task manager 230 in this example may receive estimated userintent and dialog state data from the dynamic dialog state analyzer 210,customized FAQ data from either the customized FAQ database 222 ordirectly from the customized FAQ generator 220, and/or information fromthe customized task database 139. Based on the dialog state, the FAQdata, the real-time task manager 230 may determine a next task forvirtual agent 142 to perform. Such decisions may be made based also oninformation or knowledge from the customized task database 139. Forexample, if an underlying task is to assist a chat user to find weatherinformation of a locale, the knowledge from customized task database 139for this particular task may indicate that the virtual agent or bot forthis task needs to collect information about the locale (city), date, oreven time in order to proceed to get appropriate weather information.Similarly, if the underlying task is for assisting a chat user to get arental car, the knowledge or information stored in the customized taskdatabase 139 may provide guidance as to what information a virtual agentor bot needs to collect accordingly from the chat user. For example, forthe task of identifying a rental car for a user, the information thatneeds to be collected may involve pick-up location, drop-off location,date, time, name of the user, driver license (optional), type of cardesired, price range, etc. Such information may be fed to the real-timetask manager 230 to determine what questions to ask a chat user.

A next action can be an action or a different task, selected frommultiple types of actions or tasks. For example, an action may be tocontinue to solicit additional input from the user (in order to narrowdown the specific interest of the user) by asking appropriate questions.Alternatively, an action may also be to proceed to identify anappropriate product to be recommended to the user, e.g., when it isdecided that the user input at that point is adequate to ascertain theintent. The next action may also be to proceed to a different task. Forinstance, during a session of conversation related to booking a flight,a user may ask to book a hotel room in the destination city. In thiscase, the next action is to proceed to a different task (which may behandled by a different agent, whether human or virtual agent) to takecare of the user's need for making a reservation of a hotel room.

Furthermore, real-time task manager 230 may be operating in a space thatincludes both a machine action sub-space and a human action sub-space.In the machine action sub-space, tasks/actions are handled by virtualagents. In the human action sub-space, actions/tasks are handled byhuman agents. The actions or tasks related to a dialog session may bechanneled within the same sub-space or across the two sub-spaces. Forinstance, a virtual agent in the machine sub-space may invoke anothervirtual agent in the same machine sub-space, determined based on, e.g.,the context of the dialog, the detected user intent, and/or thespecialty of other virtual agents. As another example, an action takenby a virtual agent in the machine sub-space may be to re-route to ahuman agent in the human sub-space and vice versa. The channelingbetween the two sub-spaces may be controlled based on models establishedvia machine learning. According to the present teaching, the real-timetask manager 230 may determine which action to take based on deeplearning models 225 and data obtained from knowledge database 134,publisher database 136, customized task database 139, any customerbusiness process databases, CRMs, customized skill or intent databases,any customer's enterprise document information databases and anycustomized domain-specific databases.

Real-time task manager 230 decides to continue the conversation with theuser to gather additional information, the real-time task manager 230also determines the appropriate next question to ask the user. Then thereal-time task manager 230 may send the question to the machineutterance generator 240 for generating machine utterances correspondingto the question. The machine utterance generator 240 may generatemachine utterances corresponding to the question to be presented to theuser and then present the machine utterances to the user. The generationof the machine utterances may be based on textual information or oralusing, e.g., text to speech technology.

When real-time task manager 230 determines that there has been adequateamount of information gathered to identify an appropriate product orservice for the user, real-time task manager 230 may then proceed toinvoke the recommendation engine 250 for searching an appropriateproduct or service to be recommended. Recommendation engine 250, wheninvoked, searches for product appropriate for the user based on theconversation with the user. In searching for a recommended product, inaddition to the user intent estimated during the conversation,recommendation engine 250 may also further individualize therecommendation by accessing the user's profile from user database 132.In this manner, recommendation engine 250 may individualize therecommendation based on both user's known interest (from user database132) and the user's dynamic interest (from the conversation). The searchmay yield numerous products and such searched product may be rankedbased on a machine learning model.

When real-time task manager 230 determines that the conversation withthe user involves a price that is higher than a threshold, or that theuser has a new intent associated with a domain requiring expertise otherthan that of virtual agent 142, or that the user is detected in adissatisfaction mood, the real-time task manager 230 may then invoke theagent re-router 260 for re-routing the user to a different agent. Agentre-router 260, when invoked, may re-route the user to a different agent.Depending on the context of the conversation, the re-routing agent isselected. For example, the agent re-router 260 may route the user to adifferent service virtual agent, when it is detected that what the userneeds requires expertise of the different service virtual agent.

In a different situation, agent re-router 260 may re-route the user tothe human agent 150, when, e.g., the conversation with the userindicates a situation that requires human agent involvement. Such asituation may be pre-defined or dynamically detected. For example, ifthe conversation leads to an intended transaction that involves a sum ofmoney higher than a threshold, the further handling may be re-routed toa human agent. As another example, during the conversation, it may bedetected (dynamically) that the user is dissatisfied with virtual agent142. In this case, virtual agent 142 may re-route the user to a humanagent. Similarly, if at any time, virtual agent 142 is incapable ofgathering needed information (e.g., stuck in a situation in which eitherthe user is not providing the needed information or whatever the userprovided is not comprehensible by the service virtual agent) to advancethe conversation, the user may also be re-routed to a human agent. Inyet another case, agent re-router 260 may re-direct the user'sconversation to NLU based user intent analyzer 120 to perform the NLUbased user intent analysis again and to re-route the user to acorresponding virtual agent, when e.g. virtual agent 142 detects thatthe user has a new intent associated with a different domain than thatof virtual agent 142 but cannot determine which virtual agentcorresponds to the same domain as the new intent.

However, the system of FIGS. 1 and 2 correspond to a closed system thatdoes not provide for allowing the user to interact with a service agentthat is outside of the system (e.g., a service agent that is notdeveloped by virtual agent development engine 170). Such a closed systemwould not be able to take advantage of third-party developed virtualagents.

FIG. 3A shows system 180, which allows the user to interact with one ormore external virtual agents, in accordance with one embodiment of thepresent invention. As shown in FIG. 3A, system 180 includesconversational user interface 161 which interacts with both chatbot 162and user or users 110. Chatbot 162 may be any of service agents 140 ofFIG. 1 or an external service agent. Conversational user interface 161with chatbot 162 or user 110 (or both) may be fed back to deeplearning-based dialog manager 163 to allow adaptive learning in deeplearning models in deep learning-based dialog manager 163 to improvesubsequent interactions with user 110. Deep learning-based dialogmanager 163 draws upon natural language module 164 to handle processingof natural language utterances from user 110 or for generating naturallanguage utterances for interacting with user 110. In addition, adaptivelearning input from conversational user interface 161 trains deeplearning models in deep learning-based dialog manager 163 also draw uponsystem resources 165 in its operation and training. System resources 165may include, for example, various databases (e.g., databases 130 of FIG.1), other service agents or robots (“bots”), other natural languagemodules, customer relationship modules (“CRMs”) and contact centers foradditional assistance. Chatbot 161 may be developed by interactiondesign console 166, which may be implemented, for example, by virtualagent development engine 170 of FIG. 1. As in any of the virtual agentsof FIG. 1, the deep learning models in deep-learning models in deeplearning-based dialog manager 163 may cause chatbot 162 to escalate anyappropriate cases to human agent 150.

System 180 may be implemented, for example, as an extension to thesystem of FIG. 1. In that regard, conversational user interface 161,deep learning-based dialog manager 163 and natural language 164 may bepart of NLU-based user intent analyzer 120 and service agent router 125of FIG. 1, and system resources 165 may include service agents 140 anddatabases 130 of FIG. 1. As an external virtual agent may not includeagent-specific dialog components, such as dynamic dialog analyzer 210,dialog database 212, customized FAQ generator 220, customized FAQdatabase 222, and machine utterance generator 240, these dialogcomponents may be incorporated for each external virtual agent intodeep-learning dialog manager 163. Deep learning models 225 of FIG. 2 areillustrative of deep learning models in deep-learning-based dialogmanager 163.

The operations of deep learning-based dialog manager 163 may beillustrated in part, for example, by the dynamic dialog state analyzerof Copending Application I (see, e.g., FIGS. 4A-4B and the accompanyingdetailed description in Copending Application I). The deep-learningmodels may be trained user a semi-supervised machine learning paradigm,such as illustrated in Copending Application I (see, e.g., FIGS. 3A-3Fand the accompanying detailed description in Copending Application I).FIG. 4B is a flowchart of an exemplary process for a dynamic dialogstate analyzer in a service virtual agent, according to an embodiment ofthe present teaching;

As mentioned above, system 180 allows providing an external virtualagent—as chatbot 162—to service user 110. FIGS. 3B and 3C are examplesof integrating an external virtual agent in system 160. As shown in FIG.3B, node server 201 serves as a user interface with user 110, receivinguser requests from user 110 and providing utterances and graphicaldisplay output to user 110. The user interface of node server 201 may bedriven, for example, by dialog manager 203, which may be implemented,for example, by deep learning-based dialog manager 163 of FIG. 3A. Nodeserver 201 provides each received user requests to external virtualagent 202 (e.g., an expert system, such as the IBM “Watson”). Theresponse from external virtual agent 202 may be routed through privatedata network 204 (e.g., “IBM Cloud”), for example, to dialog manager203. The interface with external virtual agent 202 may be provided, forexample, through an application programming interface (API).

FIG. 3C shows an alternative exemplary configuration for integrating anexternal virtual agent into system 180. As shown in FIG. 3C, unlike theconfiguration of FIG. 3B, dialog manager 203 does not directly controlnode server 201. Instead, responses from dialog manager 203 to the useris provided to external virtual agent 202 through private data network204 to external virtual agent 202. External virtual agent 202 thenforward the response from dialog manager 203, with or withoutformatting, to node server 201 to be provided to user 110. In thisconfiguration, external virtual agent 202 provides an API for dialogmanager 203 to send its response. As shown in FIG. 3C, node server 201interacts with external virtual agent 202 also over API interfaces.

Although only a single virtual agent is illustrated in each of theconfigurations of FIGS. 1-2 and 3A-3C, the conversational user interface(e.g., conversation user interface 161 of FIG. 3A or node server 201 ineach of FIGS. 3B-3C) of the present invention can handle multiplevirtual agents simultaneously. These virtual agents may work together ina “federated bot eco-system,” i.e., the virtual agents may cooperate torespond to a user request. In that regard, the virtual agent and theconversational user interface may be tightly-coupled (“cooperative”) orloosely-coupled (“non-cooperative”). The cooperative virtual agent(e.g., illustrated by the configurations of FIGS. 1 and 2) conforms to astandardized protocol which specifies that the virtual agent publishesits content, its conversation format, and result statistics thatcharacterizes its performance relative to a specified benchmark. Incontrast, a non-cooperative virtual agent, typically one that is engagedas an external virtual agent (e.g., the configurations of FIGS. 3B-3C)may provide as little as a request-response interface, such as a set ofAPIs.

In a federated bot eco-system, the virtual agents to serve a given usermay be selected using any one of the operational modes: (a) a browsingmodel, in which the virtual agents are selected by the user (e.g.,selecting from a menu); (b) a directory model, in which the systemselect from predetermined sets of virtual agents (“federatedconversational agents”), based on the context of an on-goingconversation with the user; (c) a message-passing model, in which a userrequest is sent out to virtual agents in the network and the agents'responses are then gathered and processed to provide the response to theuser. The message-passing model operates well in a network in whichcommunication among the agents occur in a “peer-to-peer” manner. Thus,the message-passing model is also referred to as the “peer-to-peerconversational agents.”

The operations of a system using federated conversational agents areillustrated by the flow of FIG. 4A. At step 401, the system generates aprofile for each virtual agent (i.e., information or descriptionregarding the content and services provided by the virtual agent). For agiven conversation, the virtual agents may be selected from therecognized context of the conversation (step 402). Generally, theprofiles of the candidate virtual agents to be recruited are evaluatedand ranked. Based on this ranking, a subset of virtual agents may beselected. The evaluation may be based, for example, on applying apredetermined set of metrics on the recognized context. The metrics mayfocus on both sensitivity and relevance (“recall-oriented”).

After completing the virtual agent selection, if desired, an initialrequest may be sent to the selected virtual agents (step 404). Theformat and sophistication of information in the initial request sent toeach virtual may depend on the interoperability and cooperativeness withthe conversational user interface. Typically, each selected virtualagent is expected to return a potential response, together withmeta-information regarding the response (e.g., an explanation), ifavailable (step 405). No real-time task is undertaken by the virtualagents at this juncture. From these responses, the conversational userinterface generates an integrated response to be sent to the user, usingunderlying statistics learned from experience with the selected virtualagents (step 406-409). Such statistics may relate, for example, toinverse document frequencies (“idfs”), stop lists, historical requests,and context. These statistics are generally based on relevance andcorrectness (i.e., “precision-oriented”). The virtual agents whoseresponses are incorporated in some extent or form are notified andprovided with relevant parameter values. The conversational userinterface also updates its relevant status information (step 410).

In one embodiment, in addition to sending the user request to thefederated conversational agents, the user request may be also be sent toquery a sample request database, which includes sample responsesprevious prepared or previously learned. The integrated response to theuser incorporates both responses from the federated conversationalagents, as well as from the sample response database. Machine learningtechniques may be applied to both the federated conversational agentsand the sample request database to improve their respectiveperformances.

Virtual agent selection under step 402 of FIG. 4A may be furtherillustrated by the flow chart of FIG. 4B. At step 421, all virtualagents that can be included as federal conversational agents areidentified. These agents may be then grouped according to specificcriteria (step 522). The grouping criteria may be applied automatically(steps 425-429), or manually (step 424). Automatic grouping criteriainclude (i) conversation clustering (step 425); (ii) relevantconversation distribution (step 426); (iii) rule-based selection (step427); (iv) conversation probing (step 428); and (v) statistical criteria(e.g., estimates by a classifier of the probability the virtual agentcan handle a given request; step 429).

In the above grouping criteria, the term “conversation clustering”refers to clustering or grouping of virtual agents based on similaritiesamong their historical conversations (or conversation log ortranscripts) with humans, so as to group virtual agents capable ofhandling similar human requests together for future handling of suchrequests.

The term “relevant conversation distribution” refers to using historicalinteractions between human and virtual agents—beyond merelyconversations—to extract relevant signals, such as (i) user actions(e.g., purchase, download or click signals), (ii) user satisfactionsignals (e.g., high ratings reported for conversation or sessionexperiences), (iii) domain-relevant information (e.g., relationsassociated with semantic meanings), which allow grouping of virtualagents to better satisfy user requests in specific domains (e.g.best-rated car-seller virtual agents, or 3-star rated flight-bookingvirtual agents).

The term “conversation probing” refers to a strategy most applicable fordealing with non-cooperative agents. In conversation probing, the dialogmanager uses historical user utterances or machine-generated orsimulated user utterances¹ to quest or probe a group of virtual agents.Based on the similarity of their responses and a determination on eachvirtual agent's suitability for handling the user requests at hand (e.g.some chatbots may respond with “I don't understand” or “I was not taughtto handle that request”), the virtual agents may be grouped intodifferent sets to better handle or serve future user or customerrequests. ¹ For example, using automatic grammar or rule-based naturallanguage generators

The statistical grouping criteria above may include such techniques asusing a classifier to estimate a probability that a virtual agent canhandle a given request. In such a classifier, the input signals to theclassifier may be the virtual agent's profiles, a user's or customer'sprofile, the semantic meaning of the user or customer request, andinformation from historical conversations (e.g., from conversation logsor transcripts).

Other further techniques that can be used for virtual agent selectionincludes (i) relevant request distribution and (ii) content-based botsranking and selection. Each selection mode discussed above may beadaptively improved over time using machine learning techniques (e.g.supervised learning). Content-based bots ranking and selectiontechniques may include, for example, generating for each bot summaries,descriptions, topics, terminologies, or keywords about the kind of userutterances or requests the bot can handle. Improvement in handlingfuture user requests may be achieved through more effective selection ofbots using such additional content or meta-information.

The following procedure illustrates an example of the “relevant requestdistribution” approach:

-   -   (a) build a database using training data that associates user        requests with relevant virtual agents;    -   (b) for each new user request:        -   (i) find in the database k nearest “neighbors” (i.e.,            virtual agents who have handled similar past requests); and        -   (ii) obtain a normalized relevant conversation distribution            using statistics associated with the k neighbors to obtain a            relevance metric for each potential virtual agent; and    -   (c) selecting a desired number of federated conversational        agents from the relevance metrics.

Machine learning methods may of course be used to adapt the selectionprocess. Parameters for refinement under machine learning may includedomain, security, brand, past user responses, semantics, named entityfeatures, category features, past user satisfaction features, verticalintent features, temporal features, text similarity features, hit countfeatures, and other relevant parameters.

In addition to basing on conversation context, the directory model mayalternatively be based on content. As a configuration for deployingmultiple virtual agents simultaneously to serve a user, federatedconversational agents are limited by (i) scale, (ii) hidden bots (i.e.,not all eligible virtual agents can be identified), (iii) staleinformation, (iv) network configuration (e.g., may be difficult in anetwork of short range, wireless devices), (v) need to protect personalinformation, and (vi) spamming and other undesirable conduct andincompatibility by non-cooperative virtual agents. Some of thesedisadvantages may be mitigated by other form of configuring thecollaborative virtual agents.

For a message-passing configuration (e.g., the virtual agents areprovided on a peer-to-peer network), each node of the network may be aclient, a server, or both, serving as a decentralized conversationalagent. One example of such a network of decentralized conversationalagents is illustrated in FIG. 5A (“Pure P-2-P”). These decentralizedcommunication agents may communicate, for example, using a floodalgorithm, with a predetermined time-to-live to define search horizon.In such a network each decentralized conversational agent may decidewhether or not to respond to an incoming request, according to its ownon rule-based or learning-based model.

In the network of FIG. 5A, for example, connections between nodes arerandom, with each node having knowledge only of its neighboring nodes.Such a network does not scale well. One alternative to the Pure P-2-Pnetwork of FIG. 5A is a “Central-hub” network, which is illustrated inFIG. 5B. As shown in FIG. 5B, a Central-hub network includes a hub node(labeled 531 in FIG. 5B) to which user utterances are sent. Hub node 531then routes user utterances to other decentralized conversational agentsthat can generate the desired responses. A Central-hub network issusceptible to an attack on the hub node.

Still another alternative to the Pure P-2-P and Central hub networks isthe hierarchical or “Super-peer” network, which is illustrated in FIG.5C. In one embodiment, a super-peer network may include a two-levelhierarchy of hub nodes and leaf nodes. (The hub nodes are labeled 532-a,532-b, 532-c and 532 d in FIG. 5C). In such a configuration, the leafnodes are either clients or servers and are only connect to other leafnodes through one or more hub nodes, which provide directory services tothe leaf nodes. In the hierarchical network, flooding is inefficientcommunication protocol. Instead, rather than generating a profile foreach leaf node, one may generate a profile for each neighborhood of leafnodes (i.e., the collection of leaf nodes connected to a hub node). Sucharrangement improve user utterance routing efficiency.

Other variations in the peer-to-peer architecture include (a) partialindexing over specific network resources; (b) structured peer-to-peerarchitecture using distributed hash tables, (iii) a peer-to-peer networkimplementing a gossip protocol, (iv) any architecture organizedaccording to content or user interests; (v) a peer-to-peer network withsuper peers; and (vi) a peer-to-peer network implementing reputationmanagement. These variations are known to those of ordinary skill in theart.

Those skilled in the art will recognize that the present teachings areamenable to a variety of modifications and/or enhancements. For example,although the implementation of various components described above may beembodied in a hardware device, it may also be implemented as a softwareonly solution—e.g., an installation on an existing server. In addition,the present teachings as disclosed herein may be implemented as afirmware, firmware or software combination, firmware/hardwarecombination, or a hardware, firmware or software combination.

While the foregoing has described what are considered to constitute thepresent teachings and/or other examples, it is understood that variousmodifications may be made thereto and that the subject matter disclosedherein may be implemented in various forms and examples, and that theteachings may be applied in numerous applications, only some of whichhave been described herein. It is intended by the following claims toclaim any and all applications, modifications and variations that fallwithin the true scope of the present teachings.

We claim:
 1. A virtual agent system servicing a user request from a user, comprising: a conversational user interface receiving the user request and communicating with two or more virtual agents; and a dialog manager including a natural language processing module, that directs operations of the conversational user interface, wherein the dialog manager (i) receives and analyzes the user request from the conversation user interface using the natural language processing module, (ii) causes the conversational user interface to request and to receive a response to the user request from each of the virtual agents, and (iii) integrates the received responses to the user request into an integrated response based on the natural language processing module and causes the conversational user interface to provide the integrated response to the user.
 2. The virtual agent system of claim 1, wherein at least one of the virtual agents is tightly coupled to the dialog manager.
 3. The virtual agent system of claim 1, further comprising an interactive design console and wherein the at least one of the virtual agents that is tightly coupled to the dialog manager is trained using the interactive design console.
 4. The virtual agent system of claim 1, wherein at least one of the virtual agents is loosely-coupled to the dialog manager.
 5. The virtual agent system of claim 4, wherein the at least one of the virtual agents is coupled to the dialog manager through one or more application programming interfaces.
 6. The virtual agent system of claim 1, wherein the dialog manager comprises deep-learning models trained to analyze the user request.
 7. The virtual agent system of claim 6, wherein the deep-learning models are trained through semi-supervised techniques.
 8. The virtual agent system of claim 1, further comprising one or more of the following resources drawn on by the dialog manager: (a) customer relationship modules; (b) contact centers; (c) additional virtual agents; and (d) additional natural language processing modules.
 9. The virtual agent system of claim 1, wherein each virtual agent communicates with the interactive user interface over a protocol which specifies one or more of: the virtual agent's content, conversation format, and result statistics.
 10. The virtual agent system of claim 1, wherein the virtual agents are selected by the user according to a browsing model.
 11. The virtual agent system of claim 1, wherein the dialog manager selects the virtual agents according to a directory model, based on recognizing the context of the user request.
 12. The virtual agent system of claim 11, wherein the dialog manager generates a profile for each virtual agent, the profile comprising information or description regarding the content and services provided by the virtual agent.
 13. The virtual agent system of claim 12, wherein the dialog manager selects the virtual agents by evaluating and ranking the virtual agent based, at least in part, on applying a predetermined set of metrics on the recognized context.
 14. The virtual agent system of claim 13, wherein the metrics are recall-oriented.
 15. The virtual agent system of claim 11 wherein, prior to selecting the virtual agents, the dialog manager causes the conversational user interface to send an initial request to a predetermined set of candidate virtual agents to solicit a potential response.
 16. The virtual agent system of claim 15, wherein the dialog manager integrates the potential response using precision-oriented statistics.
 17. The virtual agent system of claim 16, wherein the precision-oriented statistics comprise one or more of the following statistics: (a) inverse document frequencies, stop lists, historical requests, and context.
 18. The virtual agent system of claim 15, further comprising a sample request database, wherein the dialog manager queries the sample response database with the initial request.
 19. The virtual agent system of claim 1 wherein the virtual agents are grouped according to one or more of the following criteria: (i) conversation clustering; (ii) relevant conversation distribution; (iii) rule-based selection; (iv) conversation probing; and (v) statistical.
 20. The virtual agent system of claim 1, wherein the virtual agents are selected based on one or more of the following methods: (i) relevant request distribution and (ii) content-based bots ranking and selection.
 21. The virtual agent system of claim 20, wherein the relevant request distribution method comprises: (a) building a database using training data that associates user requests with relevant virtual agents; (b) for each new user request: (i) finding in the database k nearest “neighbors”; and (ii) obtaining a normalized relevant conversation distribution using statistics associated with the k neighbors to obtain a relevance metric for each potential virtual agent; and (c) selecting a desired number of federated conversational agents from the relevance metrics.
 22. The virtual agent system of claim 1, wherein the machine learning methods are used to adapt parameters of the dialog manager, which includes one or more of: domain, security, brand, past user responses, semantics, named entity features, category features, past user satisfaction features, vertical intent features, temporal features, text similarity features, hit count features, and other relevant parameters.
 23. The virtual agent system of claim 1, wherein the virtual agents are selected both according to a conversation context and content of the user request.
 24. The virtual agent system of claim 1, wherein the virtual agents are organized according to a message-passing model implemented in a peer-to-peer network.
 25. The virtual agent system of claim 24, wherein each node of the peer-to-peer network comprises a client, a server, or both.
 26. The virtual agent system of claim 24, wherein the peer-to-peer network implements a flood algorithm to distribute the user request to the virtual agents.
 27. The virtual agent system of claim 26, wherein each virtual agent decides whether or not to respond to the user request.
 28. The virtual agent system of claim 24, wherein the peer-to-peer network has a central-hub topology.
 29. The virtual agent system of claim 24, wherein the peer-to-peer network has a hierarchical topology including hub nodes and leaf nodes.
 30. The virtual agent system of claim 24, wherein the peer-to-peer network implements one or more of: (a) partial indexing over specific network resources; (b) structured peer-to-peer architecture using distributed hash tables, (iii) a gossip protocol, (iv) an architecture organized according to content or user interests; (v) super peers; and (vi) reputation management.
 31. In a virtual agent system servicing a user request from a user, a method comprising: in a conversational user interface, receiving the user request and communicating with two or more virtual agents; and in a dialog manager that includes a natural language processing module and that directs operations of the conversational user interface, (i) receiving and analyzing the user request from the conversation user interface using the natural language processing module, (ii) causing the conversational user interface to request and to receive a response to the user request from each of the virtual agents, and (iii) integrating the received responses to the user request into an integrated response based on the natural language processing module and causes the conversational user interface to provide the integrated response to the user.
 32. The method of claim 31, wherein at least one of the virtual agents is tightly coupled to the dialog manager.
 33. The method of claim 31, further comprising providing an interactive design console and training the at least one of the virtual agents that is tightly coupled to the dialog manager using the interactive design console.
 34. The method of claim 31, wherein at least one of the virtual agents is loosely-coupled to the dialog manager.
 35. The method of claim 34, wherein the at least one of the virtual agents is coupled to the dialog manager through one or more application programming interfaces.
 36. The method of claim 31, wherein the dialog manager comprises deep-learning models trained to analyze the user request.
 37. The method of claim 36, further comprising training the deep-learning models using semi-supervised techniques.
 38. The method of claim 31, further comprising drawing on one or more of the following resources: (a) customer relationship modules; (b) contact centers; (c) additional virtual agents; and (d) additional natural language processing modules.
 39. The method of claim 31, wherein each virtual agent communicates with the interactive user interface over a protocol which specifies one or more of: the virtual agent's content, conversation format, and result statistics.
 40. The method of claim 31, wherein the virtual agents are selected by the user according to a browsing model.
 41. The method of claim 31, further comprising the dialog manager selecting the virtual agents according to a directory model, based on recognizing the context of the user request.
 42. The method of claim 41, further comprising the dialog manager generating a profile for each virtual agent, the profile comprising information or description regarding the content and services provided by the virtual agent.
 43. The method of claim 42, further comprising the dialog manager selecting the virtual agents by evaluating and ranking the virtual agent based, at least in part, on applying a predetermined set of metrics on the recognized context.
 44. The method of claim 43, wherein the metrics are recall-oriented.
 45. The method of claim 41 further comprising, prior to selecting the virtual agents, the dialog manager causing the conversational user interface to send an initial request to a predetermined set of candidate virtual agents to solicit a potential response.
 46. The method of claim 45, further comprising the dialog manager integrating the potential response using precision-oriented statistics.
 47. The method of claim 46, wherein the precision-oriented statistics comprise one or more of the following statistics: (a) inverse document frequencies, stop lists, historical requests, and context.
 48. The method of claim 45, further comprising providing a sample request database, wherein the dialog manager queries the sample response database with the initial request.
 49. The method of claim 31, further comprising grouping the virtual agents according to one or more of the following criteria: (i) conversation clustering; (ii) relevant conversation distribution; (iii) rule-based selection; (iv) conversation probing; and (v) statistical.
 50. The method of claim 31, further comprising selecting the virtual agents based on one or more of the following methods: (i) relevant request distribution and (ii) content-based bots ranking and selection.
 51. The method of claim 50, wherein the relevant request distribution method comprises: (a) building a database using training data that associates user requests with relevant virtual agents; (b) for each new user request: (i) finding in the database k nearest “neighbors”; and (ii) obtaining a normalized relevant conversation distribution using statistics associated with the k neighbors to obtain a relevance metric for each potential virtual agent; and (c) selecting a desired number of federated conversational agents from the relevance metrics.
 52. The method of claim 31, wherein the machine learning methods are used to adapt parameters of the dialog manager, which includes one or more of: domain, security, brand, past user responses, semantics, named entity features, category features, past user satisfaction features, vertical intent features, temporal features, text similarity features, hit count features, and other relevant parameters.
 53. The method of claim 31, further comprising selecting the virtual agents both according to a conversation context and content of the user request.
 54. The method of claim 31, wherein the virtual agents are organized according to a message-passing model implemented in a peer-to-peer network.
 55. The method of claim 54, wherein each node of the peer-to-peer network comprises a client, a server, or both.
 56. The method of claim 54, wherein the peer-to-peer network implements a flood algorithm to distribute the user request to the virtual agents.
 57. The method of claim 56, wherein each virtual agent decides whether or not to respond to the user request.
 58. The method of claim 54, wherein the peer-to-peer network has a central-hub topology.
 59. The method of claim 54, wherein the peer-to-peer network has a hierarchical topology including hub nodes and leaf nodes.
 60. The method of claim 54, wherein the peer-to-peer network implements one or more of: (a) partial indexing over specific network resources; (b) structured peer-to-peer architecture using distributed hash tables, (iii) a gossip protocol, (iv) an architecture organized according to content or user interests; (v) super peers; and (vi) reputation management. 